Skip to content

Upgrade onig, to get it compiling with GCC 15 #1771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 27, 2025

Conversation

414owen
Copy link
Contributor

@414owen 414owen commented May 1, 2025

Distributions are starting to ship with GCC 15, which won't build onig. We can upgrade onig to an unreleased version, which fixes the issue. We should make a mental note to upgrade to a stable point release, when one is available (although at the moment, the onig repository is archived, so I'm not sure when that will happen).

414owen added a commit to 414owen/tokenizers that referenced this pull request May 1, 2025
Now, you can build with:

```
--no-default-features --features=fancy-regex
```

Which previously didn't work. You had to enable the `unstable_wasm`
flag.

I think using `fancy_regex` without wasm is a valid use-case, as I've
seen extremely slow build times using `onig`.
See: huggingface#1730

Onig also breaks, sometimes, with compiler updates.
See: huggingface#1771

Closes huggingface#1729
ArthurZucker pushed a commit that referenced this pull request May 27, 2025
Now, you can build with:

```
--no-default-features --features=fancy-regex
```

Which previously didn't work. You had to enable the `unstable_wasm`
flag.

I think using `fancy_regex` without wasm is a valid use-case, as I've
seen extremely slow build times using `onig`.
See: #1730

Onig also breaks, sometimes, with compiler updates.
See: #1771

Closes #1729
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be in favor of waiting a bit as this means we are introducing a non-create.io deps, preventing from publishing there AFAIK

@danieldk
Copy link
Member

Seems like there were new releases since this PR: https://github.com/rust-onig/rust-onig/releases

So an update to onig 6.5 should fix this.

@ArthurZucker
Copy link
Collaborator

@414owen if you can update it happy to merge

Distributions are starting to ship with GCC 15, which won't build `onig`.
We can upgrade `onig` to an unreleased version, which fixes the issue.
@414owen 414owen force-pushed the os/fix-onig-build-gcc-15 branch from a3ad375 to 91b2005 Compare May 27, 2025 14:20
@414owen
Copy link
Contributor Author

414owen commented May 27, 2025

👍 PR updated

@ArthurZucker ArthurZucker merged commit 909fdde into huggingface:main May 27, 2025
29 checks passed
@ArthurZucker
Copy link
Collaborator

Thanks @414owen 🤗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants